Bayesian data fusion and credit assignment in vision and fMRI data analysis

One of the most important challenges in understanding expert perception is in determining what information in a complex scene is most valuable (reliable) for a particular task, and how experts learn to exploit it. For the task of parameter estimation given multiple independent sources of data, Bayesian data fusion provides a solution to this problem that involves promoting data to a common parameter space and combining cues weighted by their reliabilities. However, for classification tasks this approach needs to be modified to find the information that most reliably distinguishes between the categories. In this paper we discuss solutions to the problem of determining the task-dependent reliability of data sources both objectively for a Bayesian decision agent, and in terms of the reliability assigned by a human observer from the performance of the observer. Modeling observers as Bayesian decision agents, solutions can be construed as a process of assigning credit to data sources based on their contribution to task performance. Applications of this approach to human perceptual data and the analysis of fMRI data will be presented.